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We introduce g-ary compressive sensing, an extension of 1-bit compressive sensing. We 
propose a novel sensing mechanism and a corresponding recovery procedure. The recovery 
properties of the proposed approach are analyzed both theoretically and empirically. Results 
in 1-bit compressive sensing are recovered as a special case. Our theoretical results suggest 
a tradeoff between the quantization parameter q, and the number of measurements m in the 
control of the error of the resulting recovery algorithm, as well its robustness to noise. 



Reconstructing signals from discrete measurements is a classic problem in signal processing. 
Properties of the signal inform the way reconstruction can be achieved from a minimal set of 
measurements. The classical Shannon sampling result ensures that band limited signals can be 
reconstructed by a linear procedure, as long as a number of linear measurements, at least twice 
the maximum frequency, is available. Modem data analysis typically requires recovering high 
dimensional signals from few inaccurate measurements. Indeed, the development of Com- 
pressed Sensing (CS) and Sparse Approximation [ ] shows that this is possible for signals with 
further structure. For example, rf-dimensional, s-sparse signals^ can be reconstructed with high 
probability through convex programming, given m ^ s \og{d/s) random linear measurements. 

Non linear measurements have been recently considered in the context of 1-bit compressive 
sensing (http : //dsp . rice . edu/lbitcs/). Here, binary (one-bit) measurements are obtained 
by applying, for example, the "sign" function-^ to linear measurements. More precisely, given 
X e M"*, a measurement vector is given by y — {yi, . . . , Um), where yi — sign{{'Wi, x)) with 
Wi ^ A/^(0, Id) independent Gaussian random vectors, for i = 1, . . . , m. It is possible to prove 
[ J ] that, for a signal x e K nM'^ (B'^ is the unit ball in M'^), the solution x,„ to the problem 



satisfies \\xm — < — tt' with probability 1—8 exp {—cS'^m), (5 > 0, as long as m > CS '^lo{KY 



[ ' ]. Here, C denotes a universal constant and lo{K) = ]Esup^gjf_jf {w,x) the Gaussian mean 
width K, which can be interpreted as a complexity measure. If X is a convex set, problem (1) 
can be solved efficiently. 

(i-dimensional signal, that is a vector in R'^, is s-sparse if only s of its components are different from zero. 
^More generally, any function : R — !> [— 1, 1], such that f!,{g9{g) > 0) can be used. 
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In this paper, borrowing ideas from signal classification studied in machine learning, we 
discuss a novel sensing strategy, based on q-ary non linear measurements, and a corresponding 
recovery procedure. 

2 q-ary Compressive Sensing 

In this section we first describe the sensing and recovery procedure (Section 2.1), then describe 
the results in the noiseless (Section 2.2) and noisy setting (Section 2.3), and finally, we sketch 
the main ideas of the proof (Section 2.4). 

2.1 Sensing and Recovery 

The sensing procedure we consider is given by a map C from KOM'^ to the g-ary , m Hamming 
cube {0, . . . , q — 1}™, where K c R'^ To define C we need the following definitions. 

Definition 1 (Simplex Coding [ ]). The simplex coding map /s S* : {0, . . . , q — 1} — > M''^^, S{j) = 
Sj , where 

for all i,j~0,...,q—l. 

Definition 2 (g-ary Quantized Measurements). Let W e M'^^^'' be a Gaussian random matrix, i.e. 
Wij ~ A/'(0, l)/or all ij. Then, Q : K nM'^ ^ {0, . . . ,q - 1}, 

Q{x) — Qw{x) — argmax {sj, Wx) , 

j=0...q-l 

is called a q-ary quantized measurement. 

Then, we can define the g-ary sensing strategy induced by non linear quantized measurements. 

Definition 3 (g-ary Sensing). Let Wi, . . . , W„i, be independent Gaussian random matrices in 
and = 1, . . . ,m as in Def. 2. The q-ary sensing is C : K nM'^ {0, . . . ,q — 1}™, 

C{x) = {Qwjx), ■ ..Qwr^{x)), 

WxeKn B'^. 

Before describing the recovery strategy we consider, we add two remarks. 

Remark 1 (Connection to 1-bit CS). Ifq = 2,W reduces to a Gaussian random vector, and 2Q{x) — 
1 — sign{Wx), so that the q-ary quantized measurements become equivalent to those considered in in 
1-bit CS. 

Remark 2 (Sensing and Embeddrngs). It can be shown that C defines an e-isometric embedding of 
{K, II -11), into {F, dn) - up-to a bias term. Here dn is the (normalized) Hamming distance, dniu, v) = 
m St=i ^ui^vi,u, V e J". This analysis is deferred to the long version of this paper. 

In this paper, we are interested in provably (and efficiently) recovering a signal x from its 
5-ary measurements y = (t/i , . . . , j/„i) = C{x). Following [ ], we consider the recovery strategy 
D : {0, . . . , g - 1}™ ^ A" n 1'* defined by, 

^ m 

D{y) = arg max — (sy . , Wiu) . (2) 
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The above problem is convex as soon as K is convex and can be solved efficiently, see Sec- 
tion 3.1. In the next section, we prove it has good recovery guarantees both in noiseless and 
noisy settings. We first add a remark. 

Remark 3 (Connection to Classification). An inspiration for considering the q-ary CS stems from an 
analogy between 1-bit compressed sensing and binary classification in machine learning. In this view, 
Definition (3) ;s related to the approach proposed for multi category classification in f j ]. Following these 
ideas, we can extend the recovery strategy (2) by considering 

Dv{y) = argmin — V V{- {sy^,W^u)), (3) 

where V is a convex, Lipchitz, non decreasing loss function y : M — > M+. Problem (2) corresponds to 
the choice V{x) = x. Other possible choices include V{x) = max(l + x, 0), V{x) = log(l + e^), and 
V{x) = e^. All these loss functions can be seen as convex relaxations of the 0-1 loss function, defined 
asV{x) — Oifx< 0, and 1 otherwise. The latter defines the mis classification risk, which corresponds 
to Hamming distance in CS, which is the natural measure of performance while learning classification 
rules. 



2.2 Recovery guarantees: Noiseless Case 

The following theorem describes the recovery guarantees for the proposed procedure, when 
applied on a signal a; in a set K of Gaussian mean width w{K). We first consider a noiseless 
scenario. 

Theorem 1. Let S > 0, and m > C6^^w{K)^. Then with probability at least 1 — 8exjp{—c6'^m), the 
solution Xm = D{y) of problem (2) satisfies, 

Win^-xf <^^^ (4) 

\/log(g) 



A proof sketch of the above result is given in Section 2.4, while the complete proof is deferred 
to the long version of the paper. Here, we add four comments. First, we note that the above 
result implies the error bound, 

\\x^-xf<C{^^^^+5), (5) 
^log(g)m 

with probability at least, 1 — 4exp(— 25^), (5 > 0. 

Second, Inequalities (4), (5) can be compared to results in 1-bit CS. For the same number 
of measurements, m > C6^^w(K)^, the error for q-ary CS is , ^ , m contrast with — i= 

^log(g) y/ f 

in the 1-bit CS [ ], at the expense of a more demanding sensing procedure. Also note that, 
for q = 2, we recover the result in 1-bit CS as a special case. Third, we see that for a given 
accuracy our results highlights a trade-off between the number of q-aiy measurements m and 
the quantization parameter q. To achieve an error e, with a memory budget of £ bits, one can 
choose TO and q so that e = 0( , ^ =), and to logofg) = i (see also section 3.2). Finally, in the 

■^mlog(g) 

following we will be interested in K being the set of s-sparse signals. Following again [ ], it is 
interesting to consider in Problem (2) the relaxation 

Ki = {u e R"^ : \\u\\^ < Ts, ||u||2 < 1}. 

With this choices, it it possible to prove that w(iiri) < C^J s log(^), and that for m > CS^^slog{^), 
the solution of the convex program (2) on Ki satisfies, \\xm ~ < ;yj=^' ^^'^ noting 
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that other choices of K are possible, for example in [ ] the set of group sparse signals (and their 
Gaussian width) are studied. 



2.3 Recovery Guarantees: Noisy Case 

Next we discuss the g-ary approach in two noisy settings, related to those considered in [ ]. 
Noise before quantization. For i = 1, . . . , m, let 

= arg max {{sj,W,x) +gj}, (6) 

j=0...g-l 

with gj independent Gaussian realization of variance . In this case, it is possible to prove 
that, for m > C5-'^w{Kf, 

\\Xm - X\\^ < , 

with probability at least 1 — 8 exp (— c(5^m). The quantization level q can be chosen to adjust to 
the noise level a for a more robust recovery of x. This result can be viewed in the perspective 
of the bit-depth versus measurement-rates perspective studied in [ ']. Here it is shown that 1-bit 
CS outperforms conventional scalar quantization. In this view, ary CS provides a new way 
to adjust the quantization parameter to the noise level. 

Inexact maximum. For i = 1, ... ,m, let yi — {x), with probability p, and yi — r with 
probability 1 — p, with r drawn uniformly at random from {0, . . . ,q — 1}. In this case, it is 
possible to prove that, for m > CS^^w{K)^, 

11- _ ||2 ^ ^ 



Vlog(9)(2p-l)' 

with probability at least 1 — 8 exp (— cJ^m). The signal x can be recovered even if half of the 
g-ary bits are flipped. 

2.4 Elements of the proofs 

We sketch the main steps in proving our results. The proof of Theorem 1 is based on: 1) deriving 
a bound in expectation, and 2) deriving a concentration result. The proof of the last step uses 
Gaussian concentration inequality extending the proof strategy in [ ]. Step 1) gives the bound 

2^ ^ ^{K) 



the proof of which is based on the following proposition. 

Proposition 1. Let E^iu) = E]y{{sj, Wu)), where 7 — Qw{x)- Then, Vu e B'', we have, 

1 2 1 

^\\u-x\\ < j^{E^{x)~ E^{u)) , 

where \{q) = E^^3((s^, 5)), and 5 - 7V(0, and^ = argmax^-^g 
Using results in empirical process theory it possible to show that 

\Ex{x) — £x{Xm.)\ ^ C . 

The bound on the expected recovery follows combining the above inequality and Proposition 1 
with the inequality, 

X{q) > C^/h^), 
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which is proved using Slepian inequality and Sudakov minoration. 

The results in the noisy settings follow from suitable estimates of X{q)- Indeed, for the noise 

before quantization case it can be proved that X{q) > C\J^^^^. For the inexact maximum case one 
has 

= Ej,,g((Sj,,g)) = 

pE( max {sj,g)) + (1 - p)E((s„ g)) > 

3 = 1. ..q 

Cpv/log(<7) + (l-p)E(.min (s„g))> 

(2p-i)cyi^. 



3 Experimental Validation 

In this section, we describe some numerical simulations in sparse recovery. Section 3.1, and 
preliminary experiments in an image recovery problem. Section 3.3. 



3.1 An Algorithm for Sparse recovery 

In our experiments, we considered the following variation of problem (2), Let = s^. Wi e 
W^,i = l...m. 

«, u L<i m ^ — ' 

i=l 

where r/ > 0. The above problem can be solved efficiently using Proximal Method [5], a solution 
can be computed via the iteration, 

m 

Ut+l = Wt + — V'^i, 
TO ^ — ' 

1=1 

Ut+1 = ProXy^{ut+i), 

1 

Where vt is the gradient step size, and Proxn acts component wise as max(l — 0)ui. The 
iteration is initialized randomly to a unit vector. 

Remark 4. The computational complexity of the sensing process depends on both m and q. Whereas, 
the computational complexity of the recovery algorithm, once computed is independent to the choice 
of q, and depends only on m and is the same as in 1-bit CS. 



3.2 Sparse Recovery 

We tested our approach for recovering a signal from from its q-ary measurements. We consid- 
ered sparse signals of dimension d generated via a Gauss-Bernoulli model. In Figure 1(a), we 
see that the reconstruction error of Xm (in blue), for varying q and to fixed, follows the theoret- 
ical bound . (in red). In Figure 1(b), we see that the reconstruction error x,n (in blue), for 

varying to and q fixed, follows the theoretical bound (in red). Figures 1(c), and 1(d) high- 
light the tradeoff between the number of measurements and the quantization parameter. For a 
precision e, and a memory budget 2^, one can choose an operating point (to, q), according to 
the theoretical bound , ^ . 

^mlog(q) 
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(a) Error ||a: — versus q, form = 70, d = 100. (b) Error — x\\^ versus m, for q = 3,d = 100. 




2C0 2m a 



(c) Theoretical bound for — versus m and (d) Error — versus m and q. 

1- 

Figure 1: q-ary Compressive Sensing: Quantization/ Number of measurements tradeoff. 
3.3 Image Reconstruction 

Then, we considered the problem of recovering an image from q-ary measurements. We used 
the standard 8— bit grayscale boat image of size 64 x 64 pixels shown in Figure 2(a). We ex- 
tracted the wavelet coefficients and performed thresholding to get a sparse signal. We normal- 
ized the resulting vector of wavelets coefficients of dimension d — 3840 to obtain a unit vector. 
Then, we performed sensing and recovery with q — 2^ (5-bit compressive sensing ) and q = 2 
(1-bit compressive sensing) for the same rn — 2048 < d. We compared the SNR performances 
of the corresponding reconstructed images in a noiseless setting (Figures 2(b)-(c)), and a noisy 
setting, considering the noise before quantization model (6), with cr = 0.8 (Figures 2(d)-(e). 
) The results confirm our theoretical: higher quantization improves the SNR, as well as the 
robustness to noise of g-ary compressive sensing. 
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Figure 2: Image recovery with g-ary CS. (a) Original image, (b) Reconstruction with no-noise: 
q = 2^, SNR = 20.2 dB. (c) Reconstruction with no-noise: q = 2, SNR = 16.2 dB. (d) Recon- 
struction with noise: q = 2^, SNR = 18.3 dB. (e) Reconstruction with noise: q = 2, SNR 15 
dB. 
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